WIP: backend Windows-native dev/test setup - #1865
Draft
derneuere wants to merge 1 commit into
Draft
Conversation
The backend normally runs in the Linux container. This adds a reproducible way to install the full pinned dependency set and run the test suite natively on a Windows dev box (Python 3.11, in-memory SQLite) -- verified end to end at 1330/1363 tests passing (remaining failures are environmental: Windows case-insensitive FS, path separators, timestamp-tie ordering, exiftool/mock test scaffolding -- not install/import failures). Native path (primary): - requirements.windows.txt: reuses every pin via -r requirements.txt / requirements.dev.txt and only swaps the Windows-friendly variants (pyvips[binary], python-magic-bin, psycopg[binary]) plus the llama-cpp-python CPU wheel index. - scripts/setup_windows.ps1: venv + pip install + winget exiftool/ImageMagick + a venv libmagic loader shim (python-magic otherwise loads Git-for-Windows' MSYS libmagic and segfaults CPython). - scripts/run_tests_windows.ps1: runs the suite against test_sqlite. Lightweight fallback (no native deps / no C++ compiler): - requirements.windows-test.txt + librephotos/settings/_win_stubs.py + test_windows.py stub the native/ML modules via a meta-path finder. .gitignore: ignore .venv*/ and .testtmp/. CLAUDE.md: document both paths. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Make the backend installable and testable natively on Windows (Python 3.11, in-memory SQLite), without Docker. Every "hostile" dependency installs from prebuilt wheels; the few that need an external binary or a C++ compiler are handled by a setup script.
Why
Lets contributors on Windows run the backend test suite without a container. Verified end-to-end: a clean install of
requirements.windows.txtinto a fresh venv +manage.py test api.tests→The ~14 failures are environmental, not install/import failures: Windows case-insensitive filesystem (e.g.
IMG_002.movvs.MOV), path separators, timestamp-tie ordering flakes, and exiftool/mock test scaffolding. Open to fixing/marking those separately if wanted.How each hostile dependency was solved (verified at pinned versions on py3.11/win_amd64)
pyvipspyvips[binary]— bundles libvips, no system installpython-magicpython-magic-bin+ a venv.pthshim. Git-for-Windows' MSYSmsys-magic-1.dllon PATH otherwise segfaults CPython onimport magic; the shim prepends the bundled libmagic dir.WandImageMagick.ImageMagick(the inno DLL build — the defaultImageMagick.Q16-HDRImsix ships no MagickWand DLLs)faiss-cpu,onnxruntime,sentencepiece,gevent,hdbscantorch/sentence_transformers/transformers/timm)torch 2.12.1+cpu)llama-cpp-python--extra-index-urlinsightfacePyExifToolOliverBetz.ExifTool(runtime binary; migration 0009 starts it)psycopgpsycopg[binary](no system libpq on Windows)gunicornfcntl); usemanage.py runserverFiles
Native path (primary)
requirements.windows.txt— DRY:-r requirements.txt+-r requirements.dev.txt+ only the Windows variantsscripts/setup_windows.ps1— venv + pip + winget binaries + libmagic shim, one commandscripts/run_tests_windows.ps1— native runner (real deps, SQLite, exiftool on PATH)Lightweight fallback (no native deps / no C++ compiler)
requirements.windows-test.txt+librephotos/settings/_win_stubs.py+test_windows.py— stub the native/ML modules via a meta-path finder so pure-Python logic tests runDocs/meta
CLAUDE.md(both paths documented),.gitignore(.venv*/,.testtmp/)Open questions for review
runserverworks without the venv.pth), or keep it dev-only?🤖 Generated with Claude Code